1. Field of the Invention
The present invention relates to a system and method of input processing for augmented reality.
2. Description of the Prior Art
Recent advances in the capabilities of videogame consoles such as the Sony® Playstation 3® console (PS3®) and other personal computers, coupled with the ubiquity of peripheral devices such as webcams and other cameras such as the Sony EyeToy®, have popularised the use of augmented reality.
Augmented reality, also known as mixed reality, combines live video input with computer graphics to enable a user to interact with an application (typically a game) though hand gesture, facial expression and/or body movement, rather than or in addition to more conventional inputs such as a mouse, keyboard or joypad.
For example, a virtual augmentation of a captured video image may comprise an augmentation image layer, comprising in turn one or more virtual entities. This layer is superposed on the captured video image, thereby combining real and virtual features. In addition, the virtual augmentation may comprise input regions on the augmented image, allowing a user to interact with the image. Typically these inputs are correlated with all or part of one or more virtual entities, but can be correlated with an element of the captured video image, for example using object recognition.
An example of such augmented reality is the well known EyePet® game, where a user interacts with a virtual pet by stroking it, or controlling virtual objects using a so-called fiduciary marker (a hand-held card with a high contrast pattern) or a Playstation Move® controller, which the PS3 can track using the EyeToy, and replace with a further virtual object on-screen. See for example http://www.youtube.com/watch?v=ORYSSQZTzbE.
However, the ability to interact seamlessly with virtual entities is limited by the precision with which the computer or console is able to relate the position of the virtual entity or entities displayed on screen with the position of the user (and in particular the user's hand) in-camera, whilst in practice the user interacts with an empty space. Conversely it is difficult for the user to interact with a virtual entity occupying that empty space if the guidance provided by the on-screen interplay of live video and augmentation is unreliable or inconsistent. Consequently there is clear scope to improve this interaction between users and virtual entities.
The present invention aims to address or mitigate the above problem.